Dynamic hand gesture recognition: An exemplar-based approach from motion divergence fields

a r t i c l e i n f o Keywords: Hand gesture recognition Divergence fields Optical flow Maximum Stable Extremal Regions Term frequency-inverse document frequency (TF-IDF) Exemplar-based approaches for dynamic hand gesture recognition usually require a large collection of gestures to achieve high-quality performance. Efficient visual representation of the motion patterns hence is very important to offer a scalable solution for gesture recognition when the databases are large. In this paper, we propose a new visual representation for hand motions based on the motion divergence fields, which can be normalized to gray-scale images. Salient regions such as Maximum Stable Extremal Regions (MSER) are then detected on the motion divergence maps. From each detected region, a local descriptor is extracted to capture local motion patterns. We further leverage indexing techniques from image search into gesture recognition. The extracted descriptors are indexed using a pre-trained vocabulary. A new gesture sample accordingly can be efficiently matched with database gestures through a term frequency-inverse document frequency (TF-IDF) weighting scheme. We have collected a hand gesture database with 10 categories and 1050 video samples for performance evaluation and further applications. The proposed method achieves higher recognition accuracy than other state-of-the-art motion and spatio-temporal features on this database. Besides, the average recognition time of our method for each gesture sequence is only 34.53 ms. Hand gestures are frequently used as intuitive and convenient ways of communications in our daily life, and the recognition of hand gestures can be widely applied in human computer interfaces, robot control, and augmented reality, etc.. Hand gestures can be conceptually divided into static gestures and dynamic gestures. Dynamic hand gestures usually provide a rich communication channel because of the incorporation of motion information, and are therefore more thoroughly investigated. The approaches to dynamic hand gesture recognition can be categorized into model-based methods and exemplar-based methods. Model-based approaches include the Hidden Markov Model and its variants [1–5], Finite State Machines [6,7], dynamic Bayesian Networks [8], and topology-preserving self-organizing networks [9]. All these approaches assume that the hand has been detected and its articulated motion is tracked. Although they have delivered promising results, the robustness of these approaches is dependent on the prior success of (frequently challenging) hand detection and motion tracking. Furthermore, it is both data intensive and computationally difficult to train these models before they can be applied in recognition. Various exemplar-based methods are therefore proposed to circumvent the …